Probability Distributions

Normal Distribution

Proof: Using the substitution $y = \frac{x - \mu}{\sigma}$, so that $dx = \sigma \, dy$, we have:

$$\int_{\mathbb{R}} f(x) \, dx = \frac{1}{\sqrt{2\pi}\sigma} \int_{-\infty}^\infty e^{-\frac{(x - \mu)^2}{2\sigma^2}} \, dx = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^\infty e^{-\frac{y^2}{2}} \, dy = \frac{1}{\sqrt{2\pi}} \cdot I,$$

where

$$I = \int_{-\infty}^\infty e^{-\frac{y^2}{2}} \, dy.$$

Note that this integral exists. Consider:

$$I^2 = \left( \int_{-\infty}^\infty e^{-\frac{x^2}{2}} \, dx \right) \left( \int_{-\infty}^\infty e^{-\frac{y^2}{2}} \, dy \right) = \int_{-\infty}^\infty \int_{-\infty}^\infty e^{-\frac{x^2 + y^2}{2}} \, dx \, dy.$$

We switch to polar coordinates, i.e., we substitute

$$x = r \cos \theta, \quad y = r \sin \theta, \quad r > 0, \; \theta \in [0, 2\pi).$$

We compute the Jacobian determinant:

$$\left| \frac{\partial(x, y)}{\partial(r, \theta)} \right| = \left| \begin{matrix} \cos\theta & -r \sin\theta \\ \sin\theta & r \cos\theta \end{matrix} \right| = r.$$

So we obtain:

$$I^2 = \int_0^\infty \int_0^{2\pi} e^{-\frac{r^2}{2}} r \, d\theta \, dr = 2\pi \int_0^\infty r e^{-\frac{r^2}{2}} \, dr.$$

Substitute $u = \frac{r^2}{2}$, so that $du = r \, dr$:

$$2\pi \int_0^\infty e^{-u} \, du = 2\pi [-e^{-u}]_0^\infty = 2\pi.$$

Hence, we conclude:

$$I = \sqrt{2\pi}, \quad \text{and therefore} \quad \int_{\mathbb{R}} f(x) \, dx = 1$$

.$\square$

Proof:

Expectation:

Substitute $z = \frac{x - \mu}{\sigma} \Rightarrow x = \sigma z + \mu$, and $dx = \sigma dz$:

$$\begin{aligned} \mathbb{E}[X] &= \int_{-\infty}^{\infty} (\sigma z + \mu) \cdot \frac{1}{\sqrt{2\pi}} e^{-z^2/2} \, dz \\ &= \sigma \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty} z \cdot e^{-z^2/2} \, dz +\mu \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} e^{-z^2/2} \, dz \\ \end{aligned}$$

First notice that $z \cdot e^{-z^2/2}$ is an odd function. Since

$$\int_{-\infty}^{\infty} |z e^{-z^2/2}| \, dz = 2 \int_0^{\infty} z e^{-z^2/2} \, dz = 2 \cdot [-e^{-z^2/2}]_0^\infty = 1,$$

we have

$$\frac{\sigma}{\sqrt{2\pi}}\int_{-\infty}^{\infty} z \cdot e^{-z^2/2} \, dz = 0.$$

(Do keep in mind: if $f$ is an odd function, then

$$\int_{\infty}^{\infty}|f(x)|dx < \infty$$

is a necessary condition of $\int_{\infty}^{\infty}f(x)dx = 0$ . Because otherwise, $f(x)=x$ is a simple counterexample.)

Moreover, we have seen in the last proof that

$$\frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} e^{-z^2/2} \, dz = 1.$$

Therefore,

$$\mathbb{E}[X] = \sigma \cdot 0 + \mu \cdot 1 = \mu.$$

Variance:

We compute:

$$\mathrm{Var}(X) = \mathbb{E}[X^2] - (\mathbb{E}[X])^2 = \int_{-\infty}^{\infty} x^2 f(x) \, dx - \mu^2$$

Again substitute $z = \frac{x - \mu}{\sigma} \Rightarrow x = \sigma z + \mu$, and $dx = \sigma dz$:

$$\begin{aligned} \int_{-\infty}^{\infty} x^2 f(x) \, dx &= \int_{-\infty}^{\infty} (\sigma z + \mu)^2 \cdot \frac{1}{\sqrt{2\pi}} e^{-z^2/2} \, dz \\ &= \int_{-\infty}^{\infty} (\sigma^2 z^2 + 2\sigma \mu z + \mu^2) \cdot \frac{1}{\sqrt{2\pi}} e^{-z^2/2} \, dz \\ &= \sigma^2 \int z^2 \phi(z) \, dz + 2\sigma\mu \int z \phi(z) \, dz + \mu^2 \int \phi(z) \, dz \\ &= \sigma^2 \cdot 1 + 2\sigma\mu \cdot 0 + \mu^2 \cdot 1 \\ &= \sigma^2 + \mu^2 \end{aligned}$$

,where

$$\phi(z) = \frac{1}{\sqrt{2\pi}} e^{-z^2/2}.$$

In addition, since $\phi’(z) = z\phi(z)$,

$$\begin{aligned} \int_{-\infty}^{\infty} z^2 \phi(z) \, dz &= \int_{-\infty}^{\infty} z \cdot (-\phi'(z)) \, dz \\ &\overset{\text{integration by parts}}{=} [-z\phi(z)]_{-\infty}^{\infty} + \int_{-\infty}^{\infty} \phi(z) \, dz \\ &= 0+1 = 1. \end{aligned}$$

Finally, we have

$$\mathrm{Var}(X) = \int_{-\infty}^{\infty} x^2 f(x) \, dx - \mu^2 = \sigma^2 + \mu^2 - \mu^2 = \sigma^2$$

.$\square$